Well, upon uploading the first version of Test::Shlomif::Harness to CPAN, I had to pick a good name for it. I brainstormed with people on Freenode's #perl channel, and eventually thought about something like Test::RunTests. Then I checked if there was already a module named "Test::Run" and guess what - there wasn't!
As a result, Test-Run is the new name and it is now on CPAN. Test::Run itself is just an API, and is not meant to be called from the command line directly (for prove or for "make test"). To do that, I also started to work on Test-Run-CmdLine.
Finally, to achieve the summary line in colour (which if you recall was my original motivation for this hacktivity), I wrote Test-Run-CmdLine-Drivers-ColorSummary. The latter is now functional and actually works. It weights in at 27 lines (excluding comments and POD), but part of it is quite sub-optimal at the moment, and may be implemented better with some refactoring of Test::Run.
Now for some issues:
I couldn't figure out how to set tags for the image I posted to Flickr. I thought I have set tags when I uploaded it, but they weren't being set and the image does not appear on a lookup of the "perl" tag.
I discovered Test::Harness' tests have not tested for the correct output (even the most elementary stuff like "All tests successful"). After my refactoring broke some stuff, I had to add some tests like that on my own. I submitted the extra tests, and talked with Andy about it in AIM, but he said that he'd rather not add these, because the output itself is "getting redone".
If you ask me, I think a test coverage should still cover these things, and be modified for the new behaviour when it is changed. This is so they'll have full coverage, and bugs that change the behaviour won't be introduced, when working or refactoring a code. Working on Test::Run has made me realise that refactoring without good test coverage can be dangerous.
At the moment, Test::Run itself depends on Class::Accessor. Is it a bad thing? Should a module like Test::Run that aims to be in the core eventually, have no dependencies except those that are distributed in perl?
On the other hand, I can lobby for the inclusion of Class::Accessor into the core. In my humble opinion, it's high time that it was included.
While I've been thinking that as far as code quality is concerned Test::Run is in pretty good shape, (much more so in comparison to Test::Harness), I often discover a piece of code there that just cries "refactor me". It is usually a function that's just too long for its own good. So I end up refactoring it. But I still appreciate the fact that I started from a working code.
Reading Parameters from the command line and from %ENV is generally something I'd like to keep out of Test::Run::Obj itself. That's because I want its entire behaviour to be controllable from its API. Now, the problem is that I'd like to people to be able to write sub-classes of Test::Run::Obj that will enhance and customise it. But these classes may have their own necessary customisations from the environment or command line. (e.g: the ::ColorSummary module should have customisable colour codes).
So the million Dollars question is whether the sub-classes should handle the CmdLine/Env handling themselves? Probably not, because a programmer may wish to control them too, from their API.
Right now I'm leaning on both sub-classing Test::Run::Obj and sub-classing an appropriate command line front-end object. The front-end sub-class will pass the arguments to the Test::Run::Obj sub-class. And there will probably be a thin wrapper module that just invokes the appropriate sub-classed front-end.
"Documentation? We don't need no stinkin' documentation!" Or at least we don't need an up-to-date documentation. At the moment, most of the documentation of Test::Run was preserved almost verbatim from that of Test::Harness and was not updated to reflect the newer changes. It needs to be updated.
Naturally, if someone writes one enhancement to Test::Run::Obj as a sub-class, and someone (else or the same) writes another one, why the two of them shouldn't be combined? This probably calls for plug-ins like I saw in Catalyst. This may mean we'll have to use Module::Pluggable and also the NEXT:: meta-class. I don't remember at which major perl 5 version it was introduced, but in this case, it may mean some enhancements will not be available in some older perls.
Finally, how you can help: first of all you can submit patches to the documentation to update it to the newer code. Or write new documentation for all them method extractions. Also, if you feel like refactoring one of the places which is still quite hairy - go for it. You can find the code in the Subversion repository. A few patches like that and you'll become a commiter.
A second thing you can do is suggest ways in which you'd like to see Test::Run (or Test::Harness) enhanced or customised. You can create drivers now, but they may become broken even in the not-so-far future, as I didn't settle on the API and the Subclassing API yet.
And last, but not least, if you find my effort noteworthy, please consider donating. The more money I receive by donations, the more financially-worthy it would be for me to work on Test-Run. If you make a donation drop me a note saying why you did it, so I can try to invest some more time on that particular cause.
Re:Core modules
Shlomi Fish on 2005-12-09T11:16:29
Hi Rafael! Thanks for your response. Now to my comments.
The general line about new core modules is : "no". So, Class::Accessor is not going into the core unless it's absolutely necessary to save the world;
I see. Well, I suppose I can create an internal Class::Accessor replacement inside Test::Run called something like Test::Run::ClassAcc or whatever, with just the functionality that I need. However, this may create problems with inheriting from Class::Accessor as well, so I'd rather avoid it if I can.
One may might as well argue that CGI.pm is not "absolutely necessary" in the Perl distribution, but it was still included because most people ended up installing it separately. Almost all of my modules end up inheriting from Class::Accessor (not to mention that Class::DBI and many other modules also depend on it), and if another candidate for a core module requires it, why not put it there while we're at it.
and I really don't see why we need another test harness in the standard Perl distribution.
Isn't CPANPLUS.pm going to be part of the core Perl distribution? And yet CPAN.pm is still present. However, CPANPLUS.pm is far superior to it. So is Test::Run in comparison to Test::Harness.
You may wish to consider my previous entry and the links there for some more information about the advantages offered by Test::Run.
On one leg, Test::Run (as opposed to Test::Harness):
- Is fully object oriented. As a result, any methods can easily be overrided by derived classes, making it possible to create customised or enhanced harnesses, while still re-using a lot of the code. Test::Harness, OTOH, is completely procedural.
- Much more modular. Many methods were extracted (most methods are much shorter) , duplicate code was eliminated, hashes were created to classes (which is why I need Class::Accessor), with some utility methods, perl formats were eliminated, etc.
- Its internal (and to some extent external) APIs are much cleaner than those of Test::Harness. The command-line/%ENV handling is being separated from the core.
The Test::Run code still has some problems, which need to be addressed before it's ready for prime-time, and I'm not saying it should be included now. But it's still heading in a better direction, and hope to see it eventually integrated into Module::Build, ExtUtils::MakeMaker and other frameworks that make use of Test::Harness.
Secondly, putting it in the core is not a critical issue at the moment. If Test::Run is good enough, then I believe many people will end up installing it until it becomes a de-facto standard. But there's still a lot of work to do (which is why I asked for help).
Replying to myself, I'd like to clarify what I said about the donations. Some people on the IRC claimed what I said sounded like some spam messages, and had some issues with the perceived message. I'd just like to say, that I'm not working on open-source code for the money, but rather out of passion like most other open source developers, who are not employed to do so, are. I will not stop working on Test::Run (or any other project in general), if I don't receive any money.
However, I still naturally need money to survive, which is why I'm asking for donations. If I find a steady job, then I will, most probably, have less time to work on open source software. If, on the other hand, I will receive enough money from donations and from text Ads on my site, then I can invest all my time on working on non-commercial endeavours. This is quite unlikely at this point, but it illustrates a point.
What I wanted to say is that if someone gives me a donation, saying he donated it out of appreciation for doing X, then I will try to invest some time enhancing it further, out of appreciation for the donation. But my hacktivity is not conditioned in me receiving donations, and never will be.
Hope everything has been clarified now.
